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Collecting data from hard-to-reach populations is a key challenge 
for research on poverty and other forms of extreme disadvantage. 
With data from the Boston Reentry Study (BRS), we document the 
extreme marginality of released prisoners and the related difficulties 
of study retention and analysis. Analysis of the BRS data yields three 
findings. First, released prisoners show high levels of “contact in- 
security,” correlated with social insecurity, in which residential 
addresses and contact information change frequently. Second, strat- 
egies for data collection are available to sustain very high rates of 
study participation. Third, survey nonresponse in highly marginal 
populations is strongly nonignorable, closely related to social and 
economic vulnerability. The BRS response rate of 94% over a 1-y 
follow-up period allows analysis of hypothetically high nonresponse 
rates. In this setting, nonresponse attenuates regression estimates in 
analyses of housing insecurity, drug use, and unemployment. These 
results suggest that in the analysis of very poor and disadvantaged 
populations, methods that maximize study participation reduce bias 
and yield data that can usefully supplement large-scale household or 
administrative data collections. 


hard-to-reach populations | poverty | study retention | survey methods | 
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ee poverty and related disadvantage pose a fundamental 
challenge for social research. Deeply disadvantaged populations 
are underenumerated and at high risk of survey nonresponse (1-3). 
If markers of disadvantage—such as unstable housing or untreated 
drug addiction—are themselves causes of nonresponse, analysis of 
disadvantage may be biased. 

Under current conditions of historically high imprisonment 
rates, incarceration has become a regular part of the hardship 
associated with American poverty (4, 5). Men and women who 
go to prison present a range of co-occurring disadvantages and 
vulnerabilities that make them difficult to contact or unwilling to 
participate in research studies. Involvement in crime and the 
criminal justice system also makes study subjects elusive for re- 
searchers. As a result, follow-up studies of the formerly incarcerated 
have suffered from high rates of attrition that are likely correlated 
with outcomes of key interest. The incarcerated population has 
been called “invisible men” because of their underenumeration in 
social survey estimates of socioeconomic well-being (6). Because of 
their deep marginality in social and economic life, invisibility in 
social science data collection also extends to those who have moved 
from incarceration to communities. 

We explore survey nonresponse in a highly disadvantaged 
population using data from the Boston Reentry Study (BRS), a 
small longitudinal survey of newly released prisoners. Earlier 
longitudinal studies of released prisoners suffered from 30% to 
60% attrition over a 1- to 2-y follow-up period (7, 8). In contrast, 
the BRS followed Massachusetts state prisoners through the first 
year of prison release, achieving a response rate of 94%. 

The unusually high response rate allows us to explore three 
main challenges to research and data collection in highly disad- 
vantaged populations. First, we document extreme social insecurity 
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in the BRS sample that is closely associated with a high level of 
contact insecurity—the frequent changing of residential addresses 
and phone numbers that makes nonresponse more likely. Second, 
we describe a variety of strategies for data collection that yield a 
high rate of study participation. Third, we show that the risk of 
survey nonresponse is associated with deep social disadvantage 
including life histories of drug addiction and mental illness. In this 
setting, nonresponse contributes to a loss of statistical precision in 
data analysis and nonresponse is nonignorable, biasing estimates of 
risk and vulnerability in the population. 

Our analysis focuses on the formerly incarcerated but the 
methods and findings reported here are generally relevant to 
other hard-to-reach populations that are unstably housed, difficult 
to contact, or otherwise weakly attached to private households. 
Although large household surveys have been central to the study of 
poverty and related topics, our results suggest the utility of 
specialized, and often small-scale, data collections for highly 
marginal populations. 


Nonresponse in Disadvantaged Populations 

Extreme poverty and other deprivation produce social insecurity 
and distrust of authorities that frustrate conventional survey 
methods. Social insecurity describes the unpredictability of daily 
life associated, for example, with housing and financial instability 
or poor health. Homelessness, unpaid phone bills, and other 
everyday catastrophes make it difficult for study subjects to be 
contacted or to keep appointments. Disability or mental illness 
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Very poor and otherwise disadvantaged populations pose a 
challenge for research. The circumstances of extreme disadvan- 
tage—homelessness, acute economic insecurity, untreated addic- 
tion or mental illness, for example—are urgent problems but also 
make research subjects elusive or unwilling. We examine study 
retention in a deeply disadvantaged sample of newly released 
prisoners, achieving a 94% response rate over five survey waves 
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for study retention and points to biases associated with non- 
response. Small-scale data collections can illuminate very harsh 
conditions of contemporary poverty by observing repeated and 
co-occurring episodes of homelessness, institutionalization, men- 
tal illness, and physical disability that are largely unobserved with 
conventional methods. 


Author contributions: B.W., A.B., and C.S. designed research; B.W., A.B., D.H., and C.S. 
performed research; B.W. and C.S. analyzed data; and B.W., D.H., and C.S. wrote 
the paper. 


Reviewers: D.H., University of California, Berkeley; and B.P., University of Texas at Austin. 
The authors declare no conflict of interest. 
‘To whom correspondence should be addressed. Email: western@wjh.harvard.edu. 


Present address: School of Criminology and Criminal Justice, Northeastern University, 
Boston, MA 02115. 


This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. 
1073/pnas. 16041381 13/-/DCSupplemental. 


PNAS Early Edition | 10f9 


wu 
_ 
= 
f& 
< 
_ 
= 
c 
= 
o 
> 
= 
= 


wn 
ery 
=] 
2 
ww 
0 
“nn 
ot 
< 
Qo 
lo} 
n 


ii 


f 


\ 


aw 


can damage normal functioning, impeding a subject’s capacity to 
arrange or complete interviews. Although study subjects may be 
willing to participate despite the obstacles associated with social 
insecurity, distrust of authorities may erode such willingness. 
Ethnographers have described distrust of authority in high-pov- 
erty field sites and the subsequent necessity of gaining trust to 
successfully collect data (9, 10). Writing about the poor residents 
in New York’s Spanish Harlem, Philippe Bourgois (9) describes 
their skepticism about assurances of confidentiality and suspi- 
cions that researchers are in fact undercover police. Distrust 
escalates to the active evasion of survey researchers where local 
residents are involved in drug dealing and other illegal activity. 

Social insecurity and distrust of authorities create problems of 
underenumeration in a cross-section, and attrition in longitudi- 
nal studies. Marginalized groups—such as the homeless, drug 
users, or sex workers—are significantly underrepresented in 
cross-sectional household surveys. Researchers have thus used 
methods such as snowball and respondent-driven sampling to 
extend coverage of hard-to-reach groups (11, 12). 

This paper focuses on attrition in longitudinal surveys. Even when 
they can be adequately sampled, disadvantaged subjects are at high 
risk of nonresponse in follow-up studies (13, 14). Researchers re- 
port low rates of follow-up among drug users (15-17), low-income 
minority parents (3, 18), women at risk for HIV infection (19), the 
homeless mentally ill (14), and youth engaged in crime and de- 
linquency (13, 20). 

Prison releasees are unusually disadvantaged. Besides their 
involvement in crime and low levels of schooling, people who 
serve time in prison suffer from high rates of mental illness and 
drug addiction (21, 22). After prison release, the formerly in- 
carcerated are often homeless or insecurely housed (23). Michigan 
parolees, for example, were found to average over six residential 
moves over about 2.5 y after prison release (24). A criminal record 
itself can contribute to residential mobility where a rental appli- 
cation is denied after a background check or where a residential 
program is mandated following a parole violation (25). Thus, 
former prisoners have been found to be more likely to be living in 
group quarters (shelters or other transitional housing) than to be 
continuously resident in stable households (26). Study subjects may 
also be deliberately elusive when on the run with outstanding arrest 
warrants, evading both researchers and police (27, 28). The stigma 
of a criminal record also contributes to unstable and undocumented 
employment and reduces access to social programs that can provide 
a source of administrative data (29, 30). In short, the formerly in- 
carcerated are weakly connected to mainstream social institutions, 
making them inaccessible to standard data collections and hard to 
follow even in specialized field studies. 

Nonresponse related to incarceration has been observed in 
medical and demographic research, and the consequences are 
likely wide-ranging in analyses of poor, prime-age, men (6, 31, 32). 
The difficulty of data collection can be seen from levels of non- 
response in studies of incarcerated populations. Table 1 lists nine 


studies since 1999 that have collected data from samples of newly 
released prison and jail inmates. The challenges of study partici- 
pation are reflected in the retention rates at exit interviews. In the 
earliest study, in New York in 1999, only around half of a sample of 
released prison and jail inmates were reinterviewed after 30 d. Most 
reentry studies record retention rates between 50% and 70%. The 
Michigan study provides an important exception, retaining 86% of a 
small sample of parolees over a 2-y follow-up period. 

The final study in Table 1, the BRS, forms the focus of this 
paper. Following a sample of Massachusetts state prisoners over 
a year after prison release, the BRS achieved a retention rate of 
91%. With a baseline sample size of N = 122, and five scheduled 
interviews for each respondent, the design included 610 inter- 
views. A total of 576 interviews were completed, yielding an 
overall response rate of 94.4%. 


Bias and Nonresponse 


High rates of nonresponse reduce the precision of estimates and 
may be a source of bias. With the small sample sizes typical of 
the studies of hard-to-reach populations, even low rates of non- 
response produce relatively large increases in sampling variance. 
Because survey nonresponse in disadvantaged populations is related 
to the conditions of disadvantage themselves, nonresponse is called 
nonignorable and can be a source of selection bias (40). Say drug 
users are difficult to contact for interviews and substantive interest 
focuses on drug use after incarceration for respondents with a history 
of addiction. For respondent i at wave f, drug use measured by y;, can 
be written as a function of x;, a dummy variable indicating drug 
addiction that is measured at baseline and is completely observed. 
Addiction may also be a risk factor for nonresponse. 
The regression of interest is 


Vit = Po + PyXi + eit, [1] 


where ej, is a random error. To see the effects of nonresponse we 
write a response indicator, Rj, that equals 1 if a survey is com- 
pleted and 0 in the case of nonresponse. A sample selection 
model writes 


fl ifm, 20 
Re={ 9 if ny <0’ 


where y;, describes the respondent’s propensity to be observed, 
Nit = O00 + O1Xj + 627; + Uit, [2] 


z; is another completely observed covariate predicting nonresponse, 
and uj is a random error. The dependence of nonresponse on the 
level of drug use y; is usually modeled by allowing the errors e; and 
Ui to be correlated. Note that Eq. 1 cannot be fit directly because yz 
is unobserved when R; =0. Analyzing just the observed data, the 
conditional expectation of the dependent variable depends on 


Table 1. Retention rates of longitudinal studies of released prisoners 


Study site Year(s) Sample size Follow-up, mo Retained at exit, % 
1. New York 1999 88 1 56.0 
2. Baltimore 2001-03 324 6 32.1 
3. Chicago 2002-03 400 16 49.5 
4. Cleveland 2002-03 424 12 69.0 
5. Houston 2004-05 676 8-10 55.9 
6. 14 states 2004-07 2,391 15 68.5 
7. Southeastern Michigan 2007-09 22 24 86.4 
8. Newark, NJ 2012-13 152 3 70.0 
9. Boston 2012-14 122 12 91.0 


Studies are described in refs. 1-9 and 33-39. 
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covariates and an adjustment factor reflecting the predicted proba- 
bility of being observed, 


E(vie|Xi,2i, Rit = 1) = Bo + Bx; +h(5o + 61%; + 6221), [3] 


where the sample selection adjustment, h, is a function whose 
form depends on assumptions about the bivariate distribution of 
ez, and uj, (41, 42). 

In the naive regression of the observed values of drug use on 
addiction—a regression of y, on x;—bias results from the correlation 
between the covariate, x;, and the residual that includes the omitted 
term, h. Below, we assess the magnitude of selection bias in a (near) 
completely observed dataset reporting estimates that exclude ob- 
servations that are at high risk of nonresponse. The key intuition is 
that if drug users with a history of addiction are difficult to contact 
for interviews, the true level of drug use among drug addicts will be 
understated by the observed data and the regression coefficients will 
be underestimated. In general, where measures of insecurity are 
outcomes of interest and insecurity is a mechanism for survey non- 
response, regression relationships will be attenuated.* 

In the study of marginal populations, risks and adversities such 
as untreated addiction, mental illness, or housing insecurity may 
all contribute to attrition from panel surveys and are also out- 
comes of key interest. In these cases, nonignorable nonresponse 
biases sample statistics but can be difficult to diagnose in the 
absence of additional information such as a refreshment sample 
or population statistics for comparison (43). 


Study Retention in a Cohort of Prison Releasees 


The BRS was designed to collect data intensively in the early 
stages of prison release. All prisoners who planned to live in the 
Boston area were eligible for the study and subjects were 
recruited with an information sheet distributed by prison staff. 
The sample included men and women, many with long histories 
of incarceration. Some respondents were released to probation 
or parole supervision, but others had completed sentences and 
returned to their communities without supervision. 

The research design involved five face-to-face interviews over a 
year. The first interview was held in prison a week before release, 
and follow-up interviews were scheduled at 1 wk after release, then 
at 2, 6, and 12 mo. The in-prison baseline interview was conducted 
by a university researcher with a staff member from the research 
unit at the Department of Correction (DOC). At the baseline in- 
terview, contact information was obtained for the first follow-up 
interview 2 wk later in the community. All follow-up interviews 
were conducted by university research staff who worked in pairs. If 
a respondent was reincarcerated, a follow-up face-to-face interview 
was conducted inside a correctional facility. 

The offense profile, criminal history, level of risk, and de- 
mographic characteristics of the BRS sample are similar to those 
for the Boston-area reentry population as a whole, suggesting 
little nonrandom selection based on observable characteristics 
(39). After release, the 1-y rate of reincarceration in the BRS 
sample (22%) was the same as that for the state’s whole reentry 
population (44). The BRS respondents represent a one-quarter 
sample returning to the Boston area from Massachusetts state 
prisons. Like the national prison population, the sample is black 
or Latino with an average age in the mid-30s, having been in- 
carcerated 2 to 3 y. 


*It is possible too, though less common in our experience, that greater social integration 
and security may contribute to nonresponse. If survey interviews are uncomfortable 
reminders of past episodes of personal crisis, respondents may be reluctant to partici- 
pate when they are doing well. Nonresponse in this case would lead to overestimates of 
social insecurity. 


*Probation is a period of community supervision determined at sentencing and adminis- 


tered by the court. Parole is a type of early release and community supervision, deter- 
mined after a period of incarceration, and administered by an executive department. 
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Characteristics of the BRS sample, reported in Table S1, place 
respondents at high risk of interview nonresponse. The sample 
was entirely urban, living in and around Boston, mostly male, 
African American and Hispanic, with very low levels of school- 
ing. Many had histories of mental illness and heavy drug use and 
had previously lived in unstable housing, either dividing time 
between multiple residences or living on the streets or in shelters 
or other group quarters. In contrast to most prison reentry 
studies, BRS respondents were not recruited from probation or 
parole agencies and over a third of the sample was released to the 
street with a completed sentence and no postrelease supervision. 
After release, nearly a quarter of the sample returned to in- 
carceration in the 1-y follow-up period, two-thirds lived in some 
kind of unstable housing, four out of five were enrolled in a food- 
stamp program, and a quarter were continuously unemployed. In 
short, like most newly released prisoners, the BRS respondents 
experienced a high level of social insecurity and were often difficult 
to contact for follow-up interviews. 

Table 2 reports the pattern of study participation in the BRS. 
Five out of 122 respondents were missed at the first follow-up 
survey and nonresponse grows to 11 respondents by the 12-mo 
exit interview. Only a minority of nonrespondents were dropouts 
in the sense of being permanently lost to follow up.* Although 
the study sustained a high response rate, data on the days to 
interview indicate the increasing difficulty of completing interviews. 
The median days to interview shows that about half the sample 
were interviewed on the scheduled follow-up day or earlier. Late- 
comers, at the 75th percentile of the days-to-interview distribution, 
got progressively later even though nonresponse did not markedly 
increase after the 2-mo follow-up. (The full distribution of the days- 
to-interview for each survey wave is shown in Fig. S1.) 

Survey methodologists distinguish between nonrespondents 
and “reluctant respondents” who are either difficult to contact or 
hesitant to consent to an interview (45). We can think of the 
latecomers, in the top quartile of the time-to-interview distri- 
bution, as reluctant respondents who are at risk for nonresponse 
without additional efforts at study retention. Latecomers and 
nonrespondents consist mostly of those who were just in- 
termittently missing or late for one or two waves (85%) and a 
smaller number who were persistently late or missing in at least 
three of the four waves (15%). 


Social Insecurity and Contact Insecurity. The social insecurity of 
hard-to-reach study subjects is linked to nonresponse by contact 
insecurity. When contact information—phone numbers and 
residential addresses—is intermittently unavailable or changes 
frequently, survey nonresponse is more likely. Table 3 reports the 
cumulative percentage of respondents who experienced contact 
insecurity by the time of each scheduled follow-up interview. 
We mainly contacted respondents by phone. Whereas most 
respondents provided the number of a family home or program 
at the baseline interview, half of the sample obtained a cell 
phone within seven days of release. Respondents often informed 
researchers of their new number when we called to schedule the 
1-wk interview. After the first week out, 23 out of 122 respon- 
dents maintained the same phone number throughout the year 
out of prison. These 23 respondents with a high level of contact 
security completed all postrelease interviews and were more 
likely to be interviewed on schedule at each wave of the survey. 
More commonly, phone status was unstable. Respondents changed 
phone numbers eight times on average over the course of the study. 
Primary phones failed to work when bills went unpaid or when a 
respondent left a residential program or returned to incarceration. 


*There were no deaths in the follow-up period, although one respondent was critically 
injured and was hospitalized near the end of the follow-up period. Another was released 
straight from prison to county jail custody for the entire year after prison release. 
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Table 2. Study participation across five waves of the BRS 


Days late for 
scheduled interview 


Respondents 75th 
Survey wave No. of interviews Response rate, % attritted, N Median percentile 
Baseline 122 100.0 _ _ _ 
1 wk 117 95.9 0 0 3 
2 mo 113 92.6 2 4 9.5 
6 mo 113 92.6 4 6 18 
12 mo 111 91.0 5 8 30 
Total 576 94.4 11 _— _— 


Survey nonresponse includes all those who could not be contacted or scheduled for a follow-up interview and 
those unreachable through incarceration or hospitalization as a percentage of those eligible to be interviewed. 
Attrition in a given wave is defined as missing the current interview and all subsequent interviews. Dashes for 


some table entries indicates not applicable. 


Some respondents obtained new phones when they started 
working or changed their phone numbers every few months. 
Others reported that regularly changing phones was a habit 
picked up when they were involved in illegal activity and feared 
their phones were being tracked. In addition, a number of 
phones went out of use when they were lost or stolen, particularly 
in the chaotic environments of homeless shelters and transitional 
housing. About a fifth of respondents were without phones at some 
point during the year-long follow-up period, and nearly all changed 
their phone numbers at least once (Table 3). 

In addition to being hard to telephone, three-quarters of re- 
spondents changed residence at least once and over half lived in 
shelters or other insecure housing. It was sometimes difficult for 
us to verify residence for respondents in transitional or other 
institutional housing because program staff could not disclose 
the names of housing occupants. Criminal justice involvement 
may also contribute to nonresponse for respondents with out- 
standing warrants or who are detained while awaiting trial. By 
the end of the follow-up period a third of respondents had been 
charged for a new offense and a fifth had spent some time back 
in custody. 


Strategies for Study Retention. In this challenging setting for study 
retention, a variety of strategies were adopted to reduce non- 
response. In a context of extreme contact insecurity, we collected 
an extensive array of supplementary contact information, main- 
tained a high level of contact between interviews, and tried to 
increase the willingness of respondents to participate. Four 
specific features of the BRS research design were intended to 
promote study participation. 

Interview incentives. Respondents were paid $50 for each in- 
terview. Incentives of about this size were previously found to 
increase participation among low-income respondents and pa- 
rolees (37, 46). Similar to another recent study of parolees in 
Michigan, BRS respondents were paid in cash at the conclusion 
of an interview (37). In the case of reincarceration, research staff 
deposited $50 into the respondent’s prison commissary account. 
At the first meeting in the community, respondents received a 
total of $100, $50 for the baseline interview in prison and $50 for 
the 1-wk follow-up. One hundred dollars was a significant pay- 
ment for people with little income, particularly for a fifth of the 
sample who were released with less than $100 in their commis- 
sary accounts. When asked why they were participating in the 
study, a quarter of respondents cited the interview payment as a 
strong incentive at the 1-wk interview. The interview incentive 
seemed to be less crucial to study retention over time. By the 
12-mo interview, only about 10% of respondents indicated they 
were participating for the money. 
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Besides the cash incentives, interviewers presented a non- 
judgmental, conversational posture in interactions with respon- 
dents. This involved unscripted probes and digressions in 
interviews motivated by a curiosity about the respondent’s life 
and welfare. The approach was echoed in unstructured com- 
munications at the conclusion of the interview (see also ref. 47). 
Respondents reported that this approach contrasted with official 
interactions with criminal justice authorities and that the attitude 
of interviewers provided an additional incentive for participa- 
tion. Reflecting the quality of the interviewer-respondent re- 
lationship, a third of the sample reported they were participating 
in the research because they were committed to completing the 
study or because they liked the interviewer. Another 20% said it 
was helpful for them to be able to share their experiences after 
incarceration.$ 
Phone check-ins and letters. Research staff also conducted regular 
phone check-ins with study respondents. Between the baseline, 
1-wk, 2-mo, and 6-mo interviews, interviewers phoned respon- 
dents every few weeks. Between the 6-mo and the 12-mo waves, 
interviewers checked in each month by phone. Phone check-ins 
were used to update the respondents’ residential information 
and to maintain contact and rapport. Phone contact involved 
both conversations and texting. Where phone contact was lost, 
we emailed or wrote letters to the respondents, often using ad- 
dress information previously provided by family members and 
friends. A strongly nonjudgmental approach in interviews com- 
bined with frequent phone check-ins were intended to build 
rapport and trust between interviewers and respondents. 
Secondary contacts and interviews. At the baseline interview, we 
asked respondents for a list of family members or friends who 
could help us to stay in contact after prison release. Similar 
protocols were used elsewhere (13, 15, 37). The list of secondary 
contacts was updated at each interview, sometimes adding new 
names or new phone numbers for existing contacts. Secondary 
contacts were particularly useful when their contact information 
was stable, when they were supportive of the respondent, and 
when we developed trust through repeated interaction. Thus, 
mothers and other older women were particularly helpful be- 
cause their own phone numbers rarely changed. Mothers and 
other supportive secondary contacts also tended to stay in touch 
with respondents, particularly in moments of crisis when the risk 
of attrition was acute. Repeated inquiries into the whereabouts 


5Repeated contact with interviewers can induce behavioral and attitudinal change (48). In 

particular, social support in BRS interviews might improve outcomes for respondents. 
Improved outcomes would be welcome news for released prisoners and policymakers, 
but insignificant differences in recidivism between the BRS sample and the state as 
whole suggests study participation yields few positive effects. 
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Table 3. Cumulative percentage of respondents experiencing 
contact insecurity by 1 wk and 2, 6, and 12 mo after 
prison release 


Percent experiencing event by 


Type of contact insecurity 1wk 2mo 6mo- 12mo 


No phone (excluding incarcerated) 5.0 8.2 12.3 21.3 


Changed phone 56.8 82.8 90.2 95.9 
Unstable/unknown residence 36.8 40.2 50.1 59.0 
Changed residence 40.2 54.1 70.5 84.4 
New charge/arraignment 0.0 5.7 13.9 33.6 
Entered prison or jail 0.8 1.6 9.0 22.1 


Data from the BRS (n = 122). Unstable residence includes staying in mul- 
tiple residences, treatment programs, transitional housing, shelters, correc- 
tional facilities, and homeless or on the street. Data on new charges and 
prison or jail stays are drawn from administrative records and thus are com- 
plete for the whole sample. 


and well-being of respondents also helped reduce suspicion and 
promoted active support of the study retention effort. 

Justice agencies and community contacts. When conventional re- 
tention strategies were exhausted, a variety of justice agencies 
and community contacts helped reestablish and maintain contact 
with the study subjects. Each week, the DOC notified the study 
team of respondents with new charges, violations, or reimpri- 
sonment. We also queried probation, parole, and police de- 
partments for phone numbers or addresses for hardest-to-reach 
respondents. Respondents had earlier consented to such contact 
and the queries were designed to be unobtrusive both for the 
agencies and the respondents. Parolees had been the focus of 
earlier reentry studies and below we study whether parole/pro- 
bation status is associated with study retention. 

For those without probation or parole supervision, we tried to 
reestablish contact through a variety of street and community 
workers operating in inner-city neighborhoods where study re- 
spondents resided. Streetworkers (gang outreach workers drawn 
from local communities) were especially helpful for reestablish- 
ing contact with younger respondents who were associated with 
gangs or other street groups. In some cases, streetworkers were 
able to provide a direct link—through a working phone number 
or current address—for a study respondent who had gone 
missing. More often, however, streetworkers connected the 


research team to a third party (say an ex-girlfriend, neighbor- 
hood acquaintance, or a former youth worker) that could ar- 
range contact with a missing respondent. When subjects dropped 
out, community contacts also offered explanations for non- 
response. For example, we learned that one subject had de- 
veloped extreme paranoia about being “set up” following a 
shooting in his area in between survey waves and was reluctant to 
meet or talk with all but a small group of trusted friends. 

Besides the four main strategies for study retention, survey re- 
cords were also linked to court data on criminal records. Admin- 
istrative data from courts and parole agencies have been valuable 
sources of information in other research on the transition from 
prison to community (25). Administrative data yield the consider- 
able advantages of large samples and long follow-up periods at low 
cost but lack the rich information on covariates and outcomes 
characteristic of primary data collection. Because administrative 
data are collected for operational and not scientific purposes, their 
validity can often only be assessed with additional observation. 
Whether a drug arrest record reflects criminal conduct or the en- 
forcement efforts of police, for example, requires more information 
than the administrative record provides. Primary and administrative 
data collection are thus complementary strategies for studying hard- 
to-reach populations. 


Social and Contact Insecurity in Qualitative Perspective. In the day- 
to-day operation of the reentry study, the retention strategies 
were applied continuously and concurrently, so much of the work 
of the data collection revolved around maximizing study partic- 
ipation. Consistent with nonignorable nonresponse, contact in- 
security was often linked to insecurity in daily life. Omar, a 
Puerto Rican man in his mid-40s, changed his phone number 17 
times over the course of the study. Omar had been a regular 
heroin user for much of his life. At 16 y old, he suffered brain 
damage after a car accident and reported it had been difficult to 
function without drugs since that time. His physical and mental 
health problems compromised his ability to maintain a steady 
phone number and to remember dates of medical appointments 
and study interviews. Still, Omar was eager to participate and on 
days when he was clear-headed and had a working cell phone he 
would sometimes call to check in. 

For the first 3 mo after incarceration, Omar resided at several 
addresses and was sometimes homeless on the streets. He 
rekindled a relationship with a former girlfriend, Jessie, who was 
just released from prison herself. Jessie had been a heavy drug 


Table 4. Multinomial logit regression results in an analysis of late and missed interviews 


compared with on-time interviews 
Model 1 


Model 2 


Independent variable Late 


Missing Late Missing 


—0.356 (0.66) 
0.474 (1.35) 
1.880** (3.30) 


Intercept 
Lag late interview 
Lag missed interview 


Age —0.048** (2.94) 
Female —0.056 (0.16) 
Addiction 0.420 (1.29) 


0.773* (2.51) 
—0.863* (2.38) 


Mental illness 
Prior unstable housing 


Unsupervised 0.253 (0.85) 

New charge 

New incarceration 

Log likelihood —250.246 
Respondent-waves, N 366 


—2.179** (2.80) 
1.213* (2.52) 
3.470** (5.95) 


0.0554 (0.13) 


—0.675 (1.17) 
0.495 (1.37) 
1.858** (3.30) 


—2.667** (3.21) 
1.270* (2.49) 
3.647** (6.40) 


—0.031 (1.50) —0.047** (2.76) —0.025 (1.12) 
—1.642 (1.78) —0.122 (0.32) —1.673* (2.05) 
1.067* (2.53) 0.415 (1.20) 0.919* (1.96) 


0.850* (2.54) 0.0172 (0.04) 


—0.818 (1.59) —0.823* (2.09) —0.926 (1.60) 
0.326 (0.72) 0.239 (0.75) 0.529 (1.14) 
0.971* (2.50) —0.144 (0.17) 


1.122 (1.91) 
—237.502 
366 


2.581* (2.38) 


Three panels of the BRS. Numbers in parentheses are z statistics. Late respondents are those in the top quartile 
of the days-to-interview distribution. SEs have been adjusted for clustering. The 1-wk survey wave is omitted to 
allow lagged measures of interview status. *P < 0.05; **P < 0.01. 
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user but was in better health and helped Omar attend appoint- 
ments, including survey interviews. She stayed with him on the 
streets and moved with Omar into a friend’s apartment 4 mo 
after his release. When Omar’s cell phone was out of service, we 
would often call Jessie, and she would hand her phone to him so 
that we could check in. 

Omar was sentenced to probation upon release but never 
reported to his probation officer due to his drug use. Just before 
his 6-mo interview, he received a warrant for not reporting. Al- 
though Omar mentioned his trouble with probation during in- 
terviews and phone calls, we received further information from 
the DOC. Omar’s warrant was dropped on the condition that he 
complete a residential drug treatment program. However, when 
we called the program a staff member reported he was no longer 
there. Thus, while Omar’s legal status was compromised and he 
was no longer living with Jessie we struggled to reach him to 
schedule the 12-mo interview. 

When Omar and Jessie were both out of contact, we would call 
Omar’s sister Lena. An older woman in her late 50s, Lena was 
16 y old when the family moved from Puerto Rico to Boston. She 
was more comfortable communicating with Spanish-speaking 
members of the research staff and had lived in public housing in 
Boston’s inner city for nearly 15 y. Lena was the most stable 
person in Omar’s life and he used her residence as his mailing 
address. Lena would get updated contact information from him 
every so often when he stopped by her place for meals. After we 
learned that Omar had left his treatment program, we contacted 
Lena, who gave us his new cell phone number. Two days later, 
we conducted Omar’s 12-mo interview. Although the interview 
took place 2.5 mo late, it provided valuable information about 
Omar’s housing insecurity, drug use, and probation late in the 
year after incarceration. 

Between the 6- and 12-mo interviews, we completed five phone 
check-ins with Omar, failed to reach him on 18 call attempts, spoke 
to his sister or girlfriend five times, and received information on his 
legal and residential status from local criminal justice agencies. His 
case points to the necessity of a range of redundant strategies to 
retain vulnerable study participants and the high risk of non- 
response for people who are struggling with housing insecurity, 
drug relapse, mental illness, and criminal involvement. 

Efforts at study retention are reported for the whole sample in 
Table S2. Dividing the sample into respondents who were con- 
sistently interviewed on time, respondents who were interviewed 
late, and those who missed interviews shows that the hardest to 
reach respondents (a third of the sample) received about 50% 
more retention effort than those who were interviewed on time. 


A Model of Reluctance and Nonresponse 


We can link respondent characteristics to nonresponse across the 
whole sample with a multinomial logit regression that takes late 
and missing interviews as response categories. Completely ob- 
served covariate characteristics can be taken from the baseline 
interview and from linked administrative data on arrests and 
incarceration. Such a model allows us to test two hypotheses. 
First, we test for an enduring propensity to nonresponse by including 
lagged dummy variables for late and missed interviews. Coefficients 
for lagged measures also suggest whether late respondents are at 
high risk of nonresponse, that is, whether reluctant respondents are 
potential nonrespondents. Second, we can also assess the association 
of lateness and nonresponse to measures of extreme disadvantage 
that are closely related to contact insecurity. These markers of social 
insecurity include histories of marginal housing, mental illness, and 
addiction, all measured at the first, in-prison, interview. Contact in- 
security is also closely related to time-varying measures of new 
charges and reincarceration. Finally, to study the risk of nonresponse 
for those who were unsupervised, a dummy variable is included for 
respondents released with completed sentences and no parole or 
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probation supervision. (Controls are also included for age, sex, race, 
and ethnicity.) 

More formally, for respondent 7 in wave t, we can write the 
probability of being late or missing, p; (j=L,M) relative to being 
interviewed on time, po, 


Pjit if t 
loo (2 ) =Aoj+ anjlit1 + a2jMit-1 +xiB 1 + Zi ;, 
Oit 


where dummy variables indicating lagged interview status are 
included for respondents who were late (J; =1) or missing 
(mj: = 1) in the prior wave, x; is a vector of time-invariant cova- 
riates, and z;; is a vector of time-varying covariates. (With lagged 
predictors, analysis is confined to the last three waves of the 
survey, ‘= 2 mo, 6 mo, and 12 mo.) Note that the multinomial 
logit model yields two sets of coefficients (j =, M), one showing 
the odds of late respondents relative to those who were on time, 
and another showing the odds of missing respondents relative to 
those who were on time. 

Multinomial logit results show that respondents who were late 
or missing in a given wave were likely to be missing at the fol- 
lowing wave, suggesting that the propensity to nonresponse is in 
part an enduring trait that is observable from wave to wave 
(Table 4). Nonresponse risk is also related to the respondent’s 
social insecurity. Respondents with histories of drug addiction 
and mental illness are only tenuously retained in the study. 
Unexpectedly, living in unstable housing before incarceration is 
associated with a relatively low risk of nonresponse, when other 
covariates are controlled. Unusually, the BRS sample also in- 
cludes a relatively large number of people who have completed 
sentences and are not supervised by probation or parole. Coeffi- 
cients for unsupervised status are relatively small and insignificant, 
suggesting the utility of the sample retention strategies that do not 
directly involve assistance from community corrections agencies. 
Estimates also show that the odds of being late for an interview are 
more than doubled by a new arrest. Respondents who return to 
prison or jail are also at relatively high risk of a noninterview, 
even in the current context with a very low nonresponse rate. The 
positive association of new charges and incarceration with the 
risk of nonresponse suggests that, in a sample with less retention 
effort and a lower response rate, survey nonresponse may be 
strongly nonignorable. 


Consequences of Reluctance and Nonresponse 

We study the effect of survey nonresponse on sample statistics 
directly by constructing a measure of the risk of nonresponse and 
examining the sensitivity of sample quantities to different hy- 
pothetical levels of missing data. Late interviews are completed 
with additional retention effort, so treating late interview sub- 
jects as potential nonrespondents yields a sample that would 
have been observed with less retention effort. A summary index 
of the risk of nonresponse can be constructed using information 
on the days late for a scheduled interview and actual nonresponse. 
Because the distribution of days late varies across waves, we 
transform the measures to percentiles, Pi, within each wave and 
assign nonrespondents to the highest percentile. An average across 
the four follow-up waves of the survey yields a time-invariant 
measure of the risk of nonresponse, P;, for each respondent. High 
values of P; indicate respondents who were consistently very late or 
who missed follow-up interviews. 

To study the sensitivity of analysis to hypothetical levels of 
nonresponse, we pool together the four follow-up waves from the 
BRS and fit logistic regressions to the pooled data using as de- 
pendent variables measures of postincarceration unstable housing, 
drug use, and unemployment. Regression analysis focuses on the 
coefficients for a history of drug addiction, mental illness, and 
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unstable housing before arrest. Regressions also control for the 
wave of the survey, age, sex, and race and ethnicity. 

A simulation experiment analyzes subsets of the data where 
observations are included in the analysis conditional on the non- 
response risk, P;. The nonresponse risk, P;, varies from 10 to 97 
with a mean of 50. We define a nonresponse threshold, T, that is 
used to drop all observations from respondent i for P; > T. With 
the reduced dataset, we estimate new regression coefficients, fr. 
These coefficients can be interpreted as the estimates that would 
have been calculated with a response rate at a level of retention 
effort at threshold T. Allowing the retention effort to vary from 
T =97,96,...,50, we can examine how coefficients change as 
nonresponse increases. With T varying over this range, the simu- 
lated nonresponse rate varies from 0% to 45%, approximately the 
range of nonresponse reported in recent longitudinal prisoner 
reentry studies. 

Fig. 1 graphs results from the simulation experiment. Results 
for each dependent variable—unstable housing, drug use, and 
unemployment—are shown down the columns. The coefficients—for 


prearrest measures of addiction, mental illness, and unstable 
housing—are shown across the rows. Hypothetical response rates 
are reported on the horizontal axis and coefficients and confidence 
intervals are shown on the vertical axis. In the model of housing 
instability, the effects of prearrest mental illness and unstable 
housing behave consistently with classical sample selection bias in 
which estimates are significantly attenuated with high rates of 
nonresponse (Fig. 1, column 1). Neither effect would be detectable 
with an 80% response rate, but both coefficients become sub- 
stantively large and significant as the response rate approaches 
100%. For postincarceration drug use, the coefficient for prearrest 
addiction grows by about a third as the response increases from 
55% to 100% (Fig. 1, column 2). In the model for unemployment, 
the coefficient for mental illness illness becomes larger as sample 
selectivity declines and sample size increases (Fig. 1, column 3). 
These results indicate how high rates of nonresponse in hard-to-reach 
populations contribute to sample selection bias, driving null results in 
the analysis of social disadvantage and vulnerability. Although 
null results are often incorrectly interpreted as effects of size 
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Fig. 1. Logistic regression coefficients for prearrest addiction, prearrest unstable housing, and probation or parole status in panel regressions on post- 


incarceration housing instability, drug use, and unemployment. Coefficients vary with the response rate sorting sample respondents from low to high risk of 
nonresponse. (Regressions fit an overdispersion parameter adjusting SEs for clustering by survey wave. Dashed lines indicate 95% confidence intervals. A zero 


coefficient is marked by the solid horizontal line.) 
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zero, the current example illustrates how null results can be 
produced by biased and inefficient data designs that arise for 
very disadvantaged groups. 


Discussion 


Three main conclusions can be drawn from this analysis. First, 
formerly incarcerated men and women show a very high level of 
contact insecurity. Most of the BRS respondents experienced 
significant residential instability, were weakly attached to stable 
households, changed phones frequently, were periodically out of 
phone contact, and in some cases returned to incarceration. 
These respondents would typically be overlooked in conven- 
tional household surveys and would experience high attrition in 
panel surveys without special measures for study retention. High 
rates of contact insecurity help explain the high rates of non- 
response reported in prior studies of prisoner reentry. 

Second, the challenges to study retention in hard-to-reach 
populations can be successfully addressed through research de- 
sign. Successful retention requires a variety of redundant strat- 
egies for minimizing attrition. Key strategies in the BRS included 
significant interview incentives, a nonjudgmental approach to 
interviews along with frequent phone check-ins that helped build 
rapport, lists of secondary contacts, and in the last resort, justice 
system and community streetworker queries. These measures 
provided strong incentives for participation and helped quickly 
find those whose contact information was lost. 

Third, contact insecurity and the risk of survey nonresponse 
were closely linked to the social risks and vulnerabilities of sci- 
entific interest. Thus, the most likely nonrespondents had his- 
tories of drug addiction and mental illness and were also more 
likely to be arrested and incarcerated after prison release. Be- 
cause the probability of noninterview is related to extreme social 
disadvantages, nonresponse is strongly nonignorable. Regression 
estimates were highly sensitive to survey nonresponse in analyses 
of housing insecurity, drug use, and unemployment. Regression 
estimates were attenuated at hypothetically high rates of non- 
response, often around 35-45%. 

Earlier studies of nonresponse in hard-to-reach populations 
report that significant resources are required to collect data from 
the most elusive respondents (13, 18). Researchers have sug- 
gested that the benefits in data quality that result from a higher 
retention rate may not be worth the cost of tracking down those 
who are most difficult to contact (18). 

Our findings point to three additional considerations when 
evaluating the benefits of a high rate of study retention. First, 
retaining the most elusive respondents does require significant 
effort, but developing a culture within the research project that 
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promotes rapport and connectedness with study subjects can 
improve retention at little extra financial cost. Second, the effort 
to sustain 100% retention likely improves the quality of inter- 
views that are completed. Where interviews are conducted in a 
climate of trust with an interviewer that a respondent has come 
to know through numerous interactions, survey responses are 
likely to be more forthcoming and complete, particularly in 
sensitive areas. Third, as our field period progressed, 100% study 
retention became as much a humanistic as a methodological 
objective. A single missing observation out of nearly 600 inter- 
views would barely increase bias or SEs, but an opportunity 
would be missed to register the voice of a respondent who has 
rarely been heard in science or policy. In very poor or socially 
marginal research sites, building trust and connection with re- 
spondents yields the benefits of accurate measurement and a 
high response rate but also gives voice to those who are largely 
invisible with conventional survey methods. 

Despite the challenges, the analysis of extreme poverty and 
other deprivation relies heavily on national surveys and administra- 
tive data (49-51). The problems of underenumeration and survey 
nonresponse are well-documented with these data (45, 52). Large 
samples offer no relief when the data design is biased. Our analysis 
shows that large-scale data collection can be usefully supplemented 
by specialized studies that focus on those who are missed by a 
household sampling frame and are at risk for attrition in a panel 
study. Counting those who are the most difficult to count holds the 
promise of observing the deepest disadvantage. In these very mar- 
ginal social spaces—where nonresponse is strongly nonignorable— 
special strategies for study retention are critical for reducing bias and 
accurately observing extreme material hardship and social insecurity. 


Materials and Methods 


Informed consent was obtained from all research subjects participating in the 
Boston Reentry Study. This research was approved under protocol number 
21354 by Harvard University’s Committee on the Use of Human Subjects. 
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